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Prefatory  Note 


This  paper  contains  a  summary  of  portions  of  the 
research  begun  under  Exploratory  Research  70,  Longi¬ 
tudinal*  Analysis  of  Aviator  Performance,  and  continued 
under  Work  Unit  PREDICT,  Predicting  Aviator  Success  in 
Training  and  Operational  Assignments.  The  research  was 
conducted  by  the  Human  Resources  Research  Organization, 
Division  No.  6  (Aviation),  Fort  Rucker,  Alabama. 


PREDICTION  OF  ARMY  AVIATOR  PERFORMANCE: 
DESCRIPTION  OF  A  DEVELOPING  SYSTEM 


Wiley  R.  Boyles  and 
James  L.  Wahlberg 


For  the  past  several  years,  a  portion  of  the  work  effort  of  HumRRO  Division  No.  6 
(Aviation)  at  Fort  Rucker,  Alabama,  has  been  devoted  to  the  development  of  a  data  bank 
containing  information  on  the  characteristics  of  Army  aviators  and  descriptions  of  their 
performance. 

Preliminary  phases  of  various  aspects  of  this  effort  have  been  described  in  earlier 
papers.  In  1967,  Prophet  reported  on  work  toward  developing  a  noncognitive  predictor  of 
performance  based  on  reactions  to  threat  of  physical  harm  (1).  In  1968,  Boyles  described 
further  stages  in  the  development  of  that  measure  (2).  Prunkl  and  Boyles  (3)  reported  on 
an  aspect  of  the  work  devoted  to  development  of  a  combat  criterion,  and  Boyd  and 
Boyles  (4)  discussed  the  use  of  attitudinal  items  from  questionnaires  as  predictors  of 
aviator  retention  in  service. 

The  purpose  of  this  paper  will  be  to  describe  the  broad  framework  in  which  these 
varied  researches  are  being  done  and  to  discuss  some  of  the  more  recent  results.  First,  we 
are  operating  in  the  context  of  a  very  expensive  training  program— expensive,  and  very 
complex  in  nature.  The  knowledges  and  psychomotor  skills  being  imparted  to  the 
students  are  complicated  technical  ones  and  involve  behaviors  that  are  very  hard  to 
predict  or  control.  Because  of  the  complexity  of  these  behaviors,  knowledge  of  the  values 
of  many  component  variables  is  necessary  for  good  description.  We  attempt  to  predict 
these  behaviors  in  a  situation  in  which  failure  to  predict  or  control  with  maximum 
efficiency  results  in  a  tremendous  amount  of  money  wasted  in  a  very  short  time.  The 
Army  has  been  training  about  6001  new  aviators  per  month  in  a  program  that  requires 
about  one  year  to  complete.  A  rough  but  conservative  estimate  of  the  cost  per  student  is 
$40,000— a  minimum  of  a  24-million  dollar  per  month  training  investment. 

One  advantage  of  the  situation,  from  the  measurement  point  of  view,  is  that  it 
furnishes  a  large  number  of  subjects  in  a  short  period  of  time.  This  affords  an  excellent 
opportunity  to  examine  a  large  number  of  variables  simultaneously,  and  anything 
approaching  an  adequate  description  of  the  events  occurring  in  this  type  of  program 
requires  that  kind  of  examination. 

A  complication,  however,  stems  from  the  fact  that  data  require  a  year  to  mature 
when  the  criterion  is  successful  completion  of  training,  and  over  two  years  to  mature 
when  the  criterion  is  job  performance.  The  job  is  a  year’s  combat  performance  in 
Vietnam  for  virtually  all  graduates.  On-site  data  collection  for  this  job  has  obvious 
drawbacks,  one  of  the  greatest  of  these  being  the  necessity  of  non-interference  in 
behavior  that  is  often  a  life  or  death  matter.  So,  job  performance  data  are  being  gathered 
as  soon  as  possible  after  completion  of  the  job. 


1  The  number  of  aviators  being  trained  has  been  reduced  since  the  time  of  this  research  because  of 
changing  military  requirements. 
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Let  me  briefly  describe  the  sequence  of  selection,  training,  and  job  performance  and 
the  data  we  are  collecting  (or  have  collected)  at  various  stages: 

For  all  potential  aviators,  the  first  step  toward  the  training  program  is  successful 
performance  on  the  primary  selection  instrument,  the  Flight  Aptitude  Selection  Test 
Battery  (FAST),  developed  by  the  Army’s  Behavior  and  Systems  Research  Laboratory.  It 
contains  a  number  of  tests  which  attempt  to  tap  several  aspects  of  cognitive  aptitude  for 
aviation  and  some  pertinent  biographical  information.  The  psychometric  characteristics  of 
the  test  battery  as  a  primary  selection  instrument  have  been  described  by  Kaplan  (5). 

In  the  PREDICT  research,  Flight  Aptitude  Selection  Test  scores  are  being  deposited 
in  the  data  bank.  Previously  reported  research  (6)  has  shown  that  these  scores  are  good 
predictors  of  whether  students  will  fail  to  complete  the  program  because  of  poor  flying 
performance. 

The  Navy  has  for  several  years  used  a  multivariate  prediction  system  in  secondary 
selection  and  their  system  is  built  around  the  use  of  similar  battery  (7).  This,  of  course, 
is  in  spite  of  the  attenuation  of  prediction  resulting  from  use  of  the  batteries  as  primary 
selectors  in  both  services. 

The  applicant  who  equals  or  surpasses  the  FAST  cut-off  score  and  meets  the  several 
other  requirements  for  entrance  into  the  program  goes  directly  to  the  primary  flight 
school  if  he  has  already  completed  Basic  Combat  Training  (BCT).  For  the  last  several 
years,  however,  about  half  the  input  to  Army  flight  training  has  consisted  of  young  men 
coming  into  the  program  from  civilian  life.  These  men  must  successfully  complete  BQT 
before  entering  the  flight  program  proper. 

The  8-week  BCT  program  involves,  among  other  things,  physical  training,  marksman¬ 
ship  training,  and  introductions  to  military  discipline  and  tactics.  Using  the  evaluations 
made  of  the  trainee  throughout  this  period,  we  have  collected  data  on  more  than  50 
variables  for  each  of  1,500  BCT  students. 

Some  preliminary  analyses  of  these  data  have  been  completed  and  have  shown 
promise  for  predicting  a  troublesome  behavior  called  “Snobird”  attrition— attrition  from 
the  training  program,  often  at  the  trainee’s  request,  occurring  after  the  completion  of 
BCT  but  prior  to  entrance  into  “preflight”  training.  In  a  high  volume  program  such  as 
Army  aviation  training,  unpredictable  fluctuations  in  weather  (scheduling  the  start  and 
finish  dates  for  training  depends  on  the  weather  and  is  necessarily  less  than  precise), 
supply  of  recruits,  and  demand  for  graduates  often  result  in  over-supplies  of  personnel  at 
various  points  during  training.  The  trainee  who  arrives  at  the  primary  training  center  and 
has  a  delay  facing  him  before  admission  to  preflight  training  is  termed  a  “Snobird.”  The 
Snobird  population  at  one  of  the  centers  has  sometimes  been  as  large  as  600  and  the 
length  of  the  delay  is  sometimes  several  weeks.  Resignations  and  other  personnel  losses 
occur  with  disturbingly  high  frequencies  during  peak  Snobird  periods,  and  identification 
of  high-probability  eliminees  prior  to  entry  into  this  status  has  potential  for  reducing 
transportation  and  training  costs. 

Prunkl  and  Tomolonis  are  conducting  the  PREDICT  study  of  the  50  BCT  variables, 
and  their  preliminary  results  show,  for  example,  a  point-biserial  correlation  of  .21 
between  scores  on  a  physical  proficiency  test  administered  early  in  BCT  and  subsequent 
Snobird  attrition.  This  relationship  is  significant  at  the  .01  level.2  The  comprehensive 
evaluation  of  their  predictors,  both  against  Snobird  attrition  and  against  flight  training 
criteria,  will  be  completed  at  a  later  date. 

Once  a  trainee  has  passed  the  BCT  and  Snobird  hurdles,  he  enters  preflight  training, 
where  quantitative  data  generated  include  academic  examination  scores  in  military 
development  subjects  and  in  aviation  subjects.  There  are  two  primary  classifications  of 
student  aviators  in  the  Army:  one  is  the  warrant  officer  candidate  (WOC),  who  is  an 

2  Prunkl,  Peter  R.,  and  Tomolonis,  Peter  C.:  Personal  Communication. 
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enlisted  man  during  training  and  becomes  an  aviation  warrant  officer  upon  graduation; 
the  other  is  the  officer  student,  who  has  received  a  commission  or  warrant  prior  to 
entering  the  aviation  program.  Although  both  are  awarded  Army  aviator  ratings  at 
graduation,  the  training  for  the  two  types  of  students  is  slightly  different.  There  are, 
therefore,  slightly  different  sets  of  information  available  for  the  two  types,  and  this  has 
implications  for  the  design  of  the  PREDICT  research. 

After  a  preflight  period  of  four  weeks  for  warrant  officer  candidates  and  approxi¬ 
mately  two  weeks  for  officer  students,  actual  flying  training  begins,  the  first,  or  primary, 
portion  of  which  is  16  weeks  long.  During  this  period,  each  student  receives  about  50 
grades  on  flying  ability.  The  warrant  officer  candidate  accumulates  27  academic  grades 
and  the  officer  student  21  during  primary  training.  All  of  these  grades  are  placed  in  the 
HumRRO  data  bank.  Upon  completion  of  primary  training,  the  student  attends  advanced 
training  at  Fort  Rucker  for  an  additional  16  weeks.  The  grades  for  each  flight  and  for 
each  academic  examination  administered  there  are  included  in  the  data  bank.  Graduation 
comes  at  the  end  of  that  period. 

Consider  now  the  task  of  the  administrator  in  this  program.  He  may  be  called  upon 
at  any  point  during  the  student’s  training  period  to  decide  whether  a  marginal  student 
should  be: 

(1)  Retained  in  the  program  and  allowed  to  continue  with  his  original  class. 

(2)  Retained,  but  given  extra  training  to  correct  his  academic,  flight  or  military 
deficiencies. 

(3)  Eliminated  from  the  program. 

Eliminating  a  student  who  could  complete  the  program  is  costly  because  a  substan¬ 
tial  investment  will  be  wasted  and  a  replacement  must  be  trained  at  further  cost. 
Retaining  a  student  who  is  highly  likely  to  fail  to  complete. the  program  is  also  wasteful. 
The  administrator,  therefore,  must  make  accurate  decisions  as  early  as  possible.  Typically, 
however,  he  is  faced  with  much  more  data  on  the  individual  than  he  can  possibly  absorb 
and  integrate  into  his  decision  in  the  short  time  allotted  him.  The  result  is  inevitably 
selection  of  portions  of  the  data  on  the  basis  of  personal  experience,  which  would  vary 
greatly  from  one  administrator  to  the  next.  Comparative  and  cumulative  validities  of 
variables  available  as  potential  predictors  are  generally  unknown. 

The  PREDICT  project  is  in  the  process  of  developing  a  multivariate  prediction 
system  which  can  use  all  the  data  available  on  the  marginal  student,  up  to  the  point  in 
training  at  which  a  decision  must  be  made  as  to  his  retention  or  elimination.  From  the 
mass  of  data,  the  system  will  select  the  most  valid  predictors,  properly  weight  them  (i.e., 
on  an  empirical  basis),  and  furnish  the  administrator  with  a  predictor  score  which  he  can 
relate  to  a  probability  table.  An  example  of  such  a  table  will  be  given  later  in  the  paper. 

Another  important  decision  that  must  be  made  by  aviation  training  administrators 
involves  selection  for  specialized  training.  An  important  current  example  is  helicopter 
gunnery  training.  About  one-fourth  of  the  aviators  are  currently  selected  for  this  training 
and  the  current  method  of  selection  is  random.  An  attempt  was  made  to  select  on  the 
basis  of  flight  grade-point  average  but  this  method  appeared  to  training  administrators,  on 
a  subjective  basis,  to  produce  poorer  results.  Now  a  large  number  of  potential  predictors 
are  available  to  compare  against  final  training  grades  in  gunnery,  and  soon  it  will  be 
possible  to  see  how  well  the  available  data  predict  this  performance. 

Even  if  this  prediction  is  reasonably  accurate,  the  job  will  be  far  from  complete. 
While  it  has  been  shown  that  quality  of  training  performance  in  military  aviation  is 
related  to  quality  of  operational  performance  (8),  the  -correlation  is  far  from  perfect. 
Thus,  simultaneously  with  the  preparation  for  validation  against  training  criteria,  a 
PREDICT  team  has  been  engaged  in  the  development  of  a  measure  of  job  performance 
(3,  9).  This  must  b.e  done  simultaneously,  because  the  requirement  for  the  job-combat 
flying— may,  hopefully,  disappear  soon. 
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We  have  begun  collecting  validation  data  (in  the  form  of  peer  nominations,  which 
seem  the  best  initial  measure  of  combat  performance)  from  aviators  returning  from 
Vietnam.  The  number  will  grow  at  the  rate  of  several  hundred  per  month. 

A  further  problem,  in  addition  to  that  of  developing  a  job  performance  measure,  is 
that  a  number  of  useful  predictor  measures  are  not  now  being  generated  in  the  Army 
system,  and  others  require  some  evaluation  before  being  used  in  the  prediction  system. 
One  of  these  is  a  peer  rating  taken  during  training.  Wahlberg  (10)  has  reported  on  an 
ongoing  study  of  peer  ratings  as  predictors  of  aviator  performance. 

The  PREDICT  research  is  moving  in  these  various  directions  simultaneously  because 
it  is  necessary,  for  good  multivariate  analysis,  to  have  all  potentially  useful  measures 
available  on  the  same  cases— and  when  the  cases  are  as  mobile  and  as  busy  as  Army 
aviators,  rapid  processing  is  essential. 

All  these  studies  are  aimed  at  a  single  goal:  to  have  a  comprehensive  description  of 
the  subject’s  characteristics  available  at  all  points  in  his  training  and  operational  develop¬ 
ment,  and  to  have  all  the  valid  predictors  available  at  a  point  in  time  for  use  in 
manpower  management  decisions.  Substantial,  though  perhaps  not  smooth,  progress  is 
being  made  toward  that  goal. 

Some  of  the  results  of  the  exploratory  work  into  prediction  of  completion  or 
attrition  during  initial  rotary  wing  training  is  given  for  a  relatively  small  sample  of  715 

subjects.  The  main  analyses  will  be  based  on  subgroup  Ns  well  in  excess  of  2,000.  The 

data  in  this  paper  represent  a  partial  evaluation  of  (a)  FAST  component  scores, 
(b)  averages  of  daily  flight  grades  at  two  early  points  in  training,  and  (c)  scores  on  a 
measure  of  voluntary  exposure  to  physical  harm  threat,  the  Background  Activities 
Inventory  (BAI).  The  criterion  variable  is  the  dichotomy  of  completion  versus  failure  to 
complete  the  flight  training  program. 

Table  1  contains  the  bivariate  correlations  between  these  variables  and  completion  of 
training  for  one  subject  classification:  Warrant  Officer  Candidates  who  came  into  the 
program  directly  after  basic  training.  This  table  reflects  data  that  are  available  during  the 

preflight  stage  of  training,  and,  thus,  there  are  no  flight  performance  variables  included  in 

this  analysis.  Table  2  lists  the  cumulative 'shrunken  multiple  Rs  for  variables  selected  by 
the  Wherry-Doolittle  procedure  for  this  point  in  training.  Table  3  is  a  probability  table 
based  on  the  regression  equations  derived  from  this  matrix. 

Table  1 

Variables  Used  in  the  Initial  Matrix 
Warrant  Officer  Candidates 
Weeks  1  Through  4 


Variables 

Fail  (rpb) 

Mean 

1.  Instrument  Comprehension 

.137 

17.72 

2.  Mechanical  Information 

.114 

14.73 

3.  Complex  Movements 

.197 

24.13 

4.  Visualization  of  Maneuvers 

.063 

18.51 

5.  Helicopter  Knowledge 

.1.41 

10.36 

6.  Stick  and  Rudder  Orientation 

.197 

25.99 

7.  Aviation  Information 

.134 

9.32 

8.  Mechanical  Functions 

.144 

15.35 

9.  Background  Activities  Inventory  (BAI) 

.070 

34.30 

Table  2 


Cumulative  R  For  Predictor  Variables 
Warrant  Officer  Candidates 
Weeks  1,  2,  3,  and  4 


Completed 

Failed 

Total 

N 

591 

124 

715 

Shrunken  Multiple  R  (Point  Biserial)  With  Complete/Attrite 

Variable 

Cumulative  R 

Complex  Movements 

.197 

Stick  and  Rudder  Orientation 

.250 

Mechanical  Functions 

.265 

Aviation  Information 

.273 

Background  Activities  Inventory  (BAI) 

.277  (R, 

<D  equivalent  .412) 

Table  3 


Probability  Table 

Enlistment  Option  Warrant  Officer  Candidates 
Weeks  1,  2,  3,  and  4 


Percent 
of  Total 

Number 

Complete 

Number 

Drop 

Approx.  Odds 

Percent 

Graduated 

For 

Against 

1050  and  Higher 

10 

68 

2 

34 

1 

97 

900-1049 

45 

289 

36 

8 

1 

89 

850-899 

18 

102 

27 

4 

1 

79 

800-849 

12 

65 

23 

3 

1 

74 

750-799 

8 

40 

15 

2.67 

1 

73 

700-749 

5 

23 

12 

2 

1 

68 

Under  700 

2 

4 

9 

1 

2 

31 

Totals 

100 

591 

124 

4.75 

1 

83 

The  first  eight  variables  shown  in  Table  1  are  eight  subtests  of  FAST,  the  primary 
selection  instrument.  The  ninth  is  the  aforementioned  measure  of  voluntary  exposure  to 
threat  of  physical  harm,  the  Background  Activities  Inventory.  This  first  analysis,  covering 
the  first  through  fourth  week  of  training,  consists  entirely  of  paper-and-pencil  tests.  As 
flying  does  not  begin  until  the  fifth  week  of  training,  the  seventh  and  eighth  weeks  of 
training,  shown  on  Table  4,  are  the  third  and  fourth  weeks,  respectively,  of  flying  activity 
for  the  students.  It  is  interesting  to  note  the  increase  in  the  multiple  R  (Table  5)  and  the 
changes  in  the  probability  table  (Table  6),  once  training  performance  measures  are 
included  in  the  matrix.  A  somewhat  arbitrary  division  has  been  made  of  flying  grade 
point  averages  in  the  second  matrix,  into  the  first  five  graded  flights  and  the  sixth 
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through  fourteenth  graded  flights.  This  is  not  entirely  arbitrary,  however,  a  student  is 
rarely  removed  from  the  program  for  flying  deficiencies  until  he  has  had  five  graded 
flights,  while  he  may  be  removed  on  any  graded  flight  thereafter. 

This  is  a  somewhat  inexact  prediction  when  compared  to  explanation  of  all  variance 
but,  compared  to  what  one  person  is  able  to  do  when  faced  with  an  individual  subject’s 
scores  on  this  same  set  of  variables,  it  is  good— good  enough,  in  fact,  to  justify  the 
construction  of  the  more  elaborate  system  which  has  been  described  in  this  paper.  This 
will  allow  the  construction  of  the  best  prediction  system  possible  within  the  constraints 
of  current  multivariate  analytic  techniques,  and  within  the  limits  of  reliability  of  the 


Table  4 

Variables  Used  in  Second  Matrix 
Warrant  Officer  Candidates 
Weeks  7  and  8 


Variables 

Fail  (rpb) 

Mean 

1.  Instrument  Comprehension 

.122 

17.85 

2.  Mechanical  Information 

.094 

14.83 

3.  Complex  Movements 

.170 

24.37 

4.  Visualization  of  Maneuvers 

.043 

18.56 

5.  Helicopter  Knowledge 

.104 

10.46 

6.  Stick  and  Rudder  Orientation 

.223 

26.03 

7.  Aviation  Information 

.100 

9.46 

8.  Mechanical  Functions 

.151 

15.41 

9.  Background  Activities  Inventory  (BAI) 

.077 

34.34 

10.  Graded  Flights  1*5 

.293 

2.62 

11.  Graded  Flights  6-14  ■  — 

.446 

2.37 

Table  5 


Cumulative  R.  For  Predictor  Variables 
Warrant  Officer  Candidates 
Weeks  7  and  8 


Completed 

Failed 

Total 

N 

590 

98 

688 

Shrunken  Multiple  R  (Point  Biseria!)  With  Complete/Attrite 

Variable 

' 

Cumulative  R 

G.P.A.  Graded  Flights  6-14 

Stick  and  Rudder  Orientation 
Mechanical  Functions 

Mechanical  Information 

Helicopter  Knowledge 

.446 

.469 

.483 

.501 

.511  (R^  equivalent  .797) 
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Table  6 


Probability  Table 

Enlistment  Option  Warrant  Officer  Candidates 
Weeks  7  and  8 


Percent 
of  Total 

Number 

Complete 

Number 

Drop 

Approx 

.  Odds 

Percent 

Graduated 

For 

Against 

1200  or  More 

04 

30 

0 

No  Failures 

100 

800-1199 

56 

369 

19 

19 

1 

95 

551-799 

27 

151 

33 

4.5 

1 

82 

400-550 

10 

35 

33 

Even 

51 

200-399 

2 

5 

10 

1 

2 

33 

199  or  Less 

<i 

0 

3 

No  Successes 

00 

Totals 

100 

590 

98 

6 

1 

86 

criteria  available.  It  is  worth  noting  that  the  Background  Activities  Inventory  (BAI), 
which  makes  a  small,  but  significant,  contribution  to  the  multiple  R  in  the  matrix  for  the 
first  four  weeks  of  training,  is  now  undergoing  item  analysis,  and  the  revised  version 
should  give  significant  improvement  of  prediction.  Further,  it  is  worth  attention  that, 
while  flight  grades  comprise  most  of  the  prediction  at  the  seventh  and  eighth  weeks,  the 
FAST  component  scores  substantially  raise  the  multiple  R. 

The  particular  FAST  subtests  selected  differ  from  those  selected  in  the  first  analysis. 
The  equivalent  biserial  values  at  both  stages  exceed  those  reported  by  the'  Navy  in  use  in 
a  system  which  has  been  highly  satisfactory  for  naval  aviation  (11).  We  are  confident  that 
our  first  comprehensive  matrices  will  give  substantially  improved  prediction.  It  is  also 
hoped  that  the  PREDICT  project  will  give  as  great  a  stability  of  prediction  as  the  Navy 
multiple  R s  which  have  held  up  extremely  well  from  year  to  year. 
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